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Dual Enrollment Programs 

Program Description 1 

Dual enrollment programs allow high school students to take college 
courses and earn college credits while still attending high school. 

Such programs, also referred to as dual credit or early college pro- 
grams, are designed to boost college access and degree attainment, 
especially for students typically underrepresented in higher educa- 
tion. Dual enrollment programs support college credit accumulation 
and degree attainment via at least three mechanisms. First, allowing 
high school students to experience college-level courses helps them 
prepare for the social and academic requirements of college while 
having the additional supports available to high school students; 
this may reduce the need for developmental coursework. Second, 
students who accumulate college credits early and consistently are 
more likely to attain a college degree. Third, many dual enrollment 
programs offer discounted or free tuition, which reduces the overall 
cost of college and may increase the number of low socioeconomic 
status students who can attend and complete college. 2 

Research 3 

The What Works Clearinghouse (WWC) identified five studies of dual 
enrollment programs that both fall within the scope of the Transition 
to College topic area and meet WWC group design standards. Two 
studies meet WWC group design standards without reservations, and 
three studies meet WWC group design standards with reservations. 

Together, these studies included 77,249 high school students across 
the United States. 

The WWC considers the extent of evidence for dual enrollment programs to be medium to large for the following 
student outcome domains— degree attainment (college), college access and enrollment, credit accumulation, com- 
pleting high school, and general academic achievement (high school). The WWC considers the extent of evidence 
for dual enrollment programs to be small for the following student outcome domains— staying in school, college 
readiness, attendance (high school), and general academic achievement (college). There were no studies that meet 
WWC group design standards in the five other domains eligible in the Transition to College topic area, so this inter- 
vention report does not report on the effectiveness of dual enrollment programs for those domains. (See the Effec- 
tiveness Summary on p. 6 for more details of effectiveness by domain.) 

Effectiveness 4 

Dual enrollment programs were found to have positive effects on students’ degree attainment (college), college 
access and enrollment, credit accumulation, completing high school, and general academic achievement (high 
school), with a medium to large extent of evidence. For the staying in high school, college readiness, and atten- 
dance (high school) domains, dual enrollment programs had potentially positive effects with a small extent of 
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evidence. Dual enrollment programs were found to have no discernible effects on general academic achievement 
(college) with a small extent of evidence. 

Table 1. Summary of findings 


Improvement index (percentile points) 

Number of Number of Extent of 


Outcome domain 

Rating of effectiveness 

Average 

Range 

studies 

students 

evidence 

Degree attainment 
(college) 

Positive effects 

+25 

+6 to +42 

5 

77,249 

Medium to large 

College access 
and enrollment 

Positive effects 

+15 

+12 to +19 

4 

67,474 

Medium to large 

Credit accumulation 

Positive effects 

+14 

+13 to +16 

2 

56,370 

Medium to large 

Completing 
high school 

Positive effects 

+7 

+5 to +9 

2 

4,052 

Medium to large 

General academic 

Positive effects 

+7 

+3 to +13 

2 

2,817 

Medium to large 


achievement 
(high school) 


Staying in 
high school 

Potentially positive effects 

+16 

na 

1 

676 

Small 

College readiness 

Potentially positive effects 

+14 

na 

1 

1,355 

Small 

Attendance 
(high school) 

Potentially positive effects 

+8 

na 

1 

1,554 

Small 

General academic 

No discernible effects 

-1 

na 

1 

455 

Small 


achievement 

(college) 

na = not applicable 


Dual Enrollment Programs February 201 7 


Page 2 



WWC Intervention Report 


Program Information 

Background 

Dual enrollment programs are collaborations between secondary and postsecondary institutions. These programs 
became common in the early 1990s and, according to the National Center for Education Statistics (NCES), there 
were more than 2 million students enrolled in dual enrollment programs by 201 1 , 5 Initially, dual enrollment programs 
were reserved for advanced students, but over time this has changed, and now average-achieving students are 
also able to participate. Dual enrollment programs are similar to other credit-based transition to college programs, 
such as Advanced Placement (AP) and International Baccalaureate (IB) programs, in that they provide rigorous 
course work and college preparation for students. In all of these programs, students are able to earn college credits 
before graduating from high school. However, dual enrollment programs allow students to take actual college 
courses, often on the postsecondary institution’s campus and taught by a college instructor, rather than a high 
school course designed to serve as part of a college-level curriculum for high school students. Dual enrollment pro- 
grams aim to help students accumulate college credits early, thereby increasing postsecondary degree attainment. 

Program details 

In the five studies that meet WWC group design standards, students were able to accumulate college credits either 
through a dual enrollment program or an early college high school program. Both of these programs allow students 
to enroll in college-level courses while still working towards their high school diploma. Students in traditional dual 
enrollment programs continue to be students in their regular high schools while also enrolling at the college or uni- 
versity where they are earning college credits. Typically, students are in grades 1 1 and 12 when they begin taking 
dual enrollment courses. Upon graduation from high school, these students can continue their college-level work 
with a reduced period of time needed to complete a postsecondary degree. 

Early college high schools are another form of dual enrollment program. Early college high schools typically offer an 
aligned curriculum that includes both high school and college courses, allowing students to earn their high school 
diplomas and up to 2 years of transferrable college credits. Some early college high schools allow students to earn 
high school diplomas and a 2-year degree as part of a 5-year program. Early college high schools tend to offer 
more formal support for students in the program as they work towards earning college-level credits. In addition to 
earning college credits offered through these schools, students are provided with more traditional college prepara- 
tion help (e.g., application assistance, financial aid application assistance, etc.) in order to reduce the barriers to 
postsecondary access and enrollment. 


Cost 

Typically, there is no cost to students enrolled in dual enrollment programs] however, the studies included in this 
review did not provide specific information about the costs of dual enrollment to the state or school district. In the 
state of Texas, high schools and postsecondary institutions are able to receive more funds from the state in order to 
cover the costs of dual enrollment programs. 6 Additionally, the early college high school initiative was funded by the 
Bill and Melinda Gates Foundation along with other corporations and foundations (13 partner organizations). They 
have provided $806 million worth of grants to support early college high schools throughout the United States. 7 
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Research Summary 


The WWC identified 35 eligible studies that investigated the effects 
of dual enrollment programs for high school students. An additional 
27 studies were identified but do not meet WWC eligibility criteria for 
review in this topic area. Citations for all 62 studies are in the Refer- 
ences section, which begins on p. 12. 


Table 2. Scope of reviewed research 


Grades 

9, 10, 11, 12 

Delivery method 

Individual, Whole dass, 
Whole school 

Program type 

Practice, Curriculum 


The WWC reviewed 35 eligible studies against group design standards. 

Two studies (Berger, Garet, Hoshen, Knudson, & Turk-Bicakci, 2014; 

Edmunds, Unlu, Glennie, Bernstein, Fesler, Furey, & Arshavsky, 2015) are randomized controlled trials that meet 
WWC group design standards without reservations, and three studies (An, 2013; Giani, Alexander, & Reyes, 2014; 
Struhl & Vargas, 2012) are randomized controlled trials or use quasi-experimental designs that meet WWC group 
design standards with reservations. Those five studies are summarized in this report. Thirty studies do not meet 
WWC group design standards. 


Summary of studies meeting WWC group design standards without reservations 

Berger et al. (2014) conducted a randomized controlled trial that examined the effectiveness of early college high 
schools (ECHS). Students were assigned to participate in either an ECFIS or a comparison group school via a lot- 
tery system. There were a total of 1,044 students in 10 ECFIS in the intervention group and 1,414 students spread 
across 272 high schools in the comparison group. Early college high schools aim to provide underserved students 
with exposure to, and support in, college while they are still in high school. Early college high schools partner with 
colleges and universities to offer all students an opportunity to earn an associate degree or up to 2 years of college 
credits toward a bachelor’s degree during high school at no or low cost to the students. 

Edmunds et al. (2015) conducted a randomized controlled trial to examine the effectiveness of early college high 
schools (ECHS). The study took place in schools in several districts in North Carolina. The sample included stu- 
dents who had applied to early college high schools in North Carolina in the eighth grade. Using a lottery system, 
these students were either offered enrollment at an Early College High school in grade 9 or not. The total sample 
included 1 ,651 students. The 19 early college high schools in the study targeted students traditionally under- 
represented in college— that is, first-generation college students, those from low-income families, and/or members 
of underrepresented racial or ethnic minority groups. The ECHS in this study typically enrolled fewer than 400 stu- 
dents and included grades 9-13 or 9-12. 

Summary of studies meeting WWC group design standards with reservations 

An (2013) conducted a quasi-experimental study to examine the effectiveness of dual enrollment programs. Dual 
enrollment programs provide students with a way to enroll in college and earn college credits while still in high 
school. The study sample was drawn from the fourth follow-up of the National Education Longitudinal Study (NELS) 
of 1988, which was conducted in 2000. The intervention group was comprised of students in the 2000 NELS 
follow-up survey who attended postsecondary school and who had participated in a dual enrollment program while 
in high school, which included 880 students. The comparison group was comprised of NELS respondents who 
attended a postsecondary school who participated in other high school programs (e.g., traditional and Advanced 
Placement programs), but not dual enrollment, which included 7,920 students. The author used a propensity score 
matching process to select an observationally equivalent comparison group. A total of 8,800 students across the 
United States were used for the study sample. 

Giani et al. (2014) used a quasi-experimental design to examine the effectiveness of taking dual credit courses. 

The sample included one cohort of ninth graders who had been enrolled in public high schools in Texas during 
the 2000-01 school year. The intervention group consisted of students who took and passed one or more dual 
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credit courses in their junior or senior year. Propensity score matching was used to create matched groups. The 
comparison group was created from the sample of students who were enrolled in districts which did not offer any 
dual-credit courses to minimize self-selection bias. Both the intervention and comparison group samples were 
comprised of 15,716 students. Students were tracked through the 2009-10 school year. 

Struhl and Vargas (2012) used a quasi-experimental design to examine the effectiveness of dual enrollment partici- 
pation for improving postsecondary outcomes. The study selected high school seniors in the 2003-04 academic 
year who had remained in the same school district all 4 years of high school, and tracked their progress over the 
course of 6 years. Students who participated in dual enrollment were compared to students who did not participate 
in dual enrollment. Propensity score matching was used to create the intervention and comparison groups. A total 
of 132,772 students graduated in the 2004 academic year. Achievement, income, and race variables were used for 
matching to select students for the intervention and comparison groups. After matching, a total of 16,454 students 
were selected for the intervention and comparison groups. 
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Effectiveness Summary 

The WWC review of dual enrollment programs for the Transition to College topic area includes outcomes in 13 
domains: general academic achievement (middle school), general academic achievement (high school), attendance 
(middle school), attendance (high school), college readiness, staying in high school, progressing in high school, com- 
pleting high school, college access and enrollment, credit accumulation, general academic achievement (college), 
degree attainment (college), and labor market. The six studies of dual enrollment programs that meet WWC group 
design standards reported findings in nine of the 13 domains: (a) degree attainment (college), (b) college access and 
enrollment, (c) credit accumulation, (d) completing high school, (e) general academic achievement (high school), 

(f) staying in high school, (g) college readiness, (h) attendance (high school), and (i) general academic achievement 
(college). The findings below present the authors’ estimates and WWC-calculated estimates of the size and statisti- 
cal significance of the effects of dual enrollment programs for students. Additional comparisons are presented as 
supplemental findings in Appendix D. The supplemental findings do not factor into the intervention’s rating of effec- 
tiveness. For a more detailed description of the rating of effectiveness and extent of evidence criteria, see the WWC 
Rating Criteria on p. 43. 

Summary of effectiveness for the degree attainment (college) domain 

Two studies that meet WWC group design standards without reservations and three studies that meet WWC group 
design standards with reservations reported findings in the degree attainment (college) domain. 

Berger et al. (2014) compared student graduation rates for early college high school participants to students in the 
comparison group. The authors reported, and the WWC confirmed, that there was a statistically significant differ- 
ence between intervention and comparison group students on high school graduation rates. The WWC character- 
izes this finding as a statistically significant positive effect. 

Edmunds et al. (2015) reported the percentage of students in the intervention and comparison groups who earned 
postsecondary credentials. The authors reported, and the WWC confirmed, that there was a statistically significant 
positive effect on degree attainment (college) for students who participated in the program. The WWC characterizes 
this finding as a statistically significant positive effect. 

An (2013) reported on the percentage of students in the intervention and comparison groups who obtained any col- 
lege degree. The author reported, and the WWC confirmed, that there was a statistically significant positive effect 
on degree attainment (college) for students who participated in the program. The WWC characterizes this finding as 
a statistically significant positive effect. 

Giani et al. (2014) reported on the number of students in the intervention group who completed a postsecond- 
ary degree or certificate as compared to students in the comparison group. The authors reported, and the WWC 
confirmed, that there was a statistically significant positive effect on degree attainment (college) for students who 
participated in the program. The WWC characterizes this finding as a statistically significant positive effect. 

Struhl and Vargas (2012) reported on the number of students in the intervention group who earned a college degree 
versus students in the comparison group who earned a college degree. The authors reported, and the WWC 
confirmed, that there was a statistically significant positive effect on degree attainment (college) for students who 
participated in the program. The WWC characterizes this finding as a statistically significant positive effect. 

Thus, for the degree attainment (college) domain, five studies found statistically significant positive effects. This 
results in a rating of positive effects, with a medium to large extent of evidence. 
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Table 3. Rating of effectiveness and extent of evidence for the degree attainment (college) domain 


Rating of effectiveness 

Criteria met 

Positive effects 

Strong evidence of a positive effect 
with no overriding contrary evidence. 

In the five studies that reported findings, the estimated impact of the intervention on outcomes in the degree 
attainment (college) domain was positive, because five studies show statistically significant positive effects, 
and no studies show statistically significant or substantively negative effects. 

Extent of evidence 

Criteria met 

Medium to large 

Five studies that included 77,249 students in schools across the United States reported evidence of effective- 
ness in degree attainment (college) domain. 


Summary of effectiveness for the college access and enrollment domain 

Two studies that meet WWC group design standards without reservations and two studies that meet WWC group 
design standards with reservations reported findings in the college access and enrollment domain. 

Berger et al. (2014) reported the percentage of students who enrolled in college. The authors reported, and the WWC 
confirmed, that there was a statistically significant difference between intervention and comparison group students 
on college enrollment rates. The WWC characterizes this finding as a statistically significant positive effect. 

Edmunds et al. (2015) reported, based on survey data collected, the number of students who planned to attend 
a 4-year college after high school graduation. The authors reported, and the WWC confirmed, that there was a 
statistically significant difference between intervention and comparison group students on plans to attend a 4-year 
college. The WWC characterizes this finding as a statistically significant positive effect. 

Giani et al. (2014) reported the number of students who enrolled in any postsecondary institution following high 
school graduation. The authors reported, and the WWC confirmed, that there was a statistically significant differ- 
ence between intervention and comparison group students on college enrollment rates. The WWC characterizes 
this finding as a statistically significant positive effect. 

Struhl and Vargas (2012) reported the number of students who enrolled in a postsecondary institution following high 
school graduation. The authors reported, and the WWC confirmed, that there was a statistically significant differ- 
ence between intervention and comparison group students on postsecondary enrollment rates. The WWC charac- 
terizes this finding as a statistically significant positive effect. 

Thus, for the college access and enrollment domain, four studies found statistically significant positive effects. 
Dual enrollment programs have a positive effect on college access and enrollment with a medium to large extent 
of evidence. 


Table 4. Rating of effectiveness and extent of evidence for the college access and enrollment domain 


Rating of effectiveness 

Criteria met 

Positive effects 

Strong evidence of a positive effect 
with no overriding contrary evidence. 

In the four studies that reported findings, the estimated impact of the intervention on outcomes in the staying in 
college access and enrollment domain was positive, because four studies show statistically significant positive 
effects, and no studies show statistically significant or substantively negative effects. 

Extent of evidence 

Criteria met 

Medium to large 

Four studies that included 67,474 students in schools across the United States reported evidence of effective- 
ness in the college access and enrollment domain. 
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Summary of effectiveness for the credit accumulation domain 

Table 5. Rating of effectiveness and extent of evidence for the credit accumulation domain 


Rating of effectiveness 

Criteria met 

Positive effects 

Strong evidence of a positive effect 
with no overriding contrary evidence. 

In the two studies that reported findings, the estimated impact of the intervention on outcomes in the credit accu- 
mulation domain was positive, because two studies show statistically significant positive effects, and no studies 
show statistically significant or substantively negative effects. 

Extent of evidence 

Criteria met 

Medium to large 

Two studies that included 64,340 students in schools across the United States reported evidence of effective- 
ness in the credit accumulation domain. 


Two studies that meet WWC group design standards with reservations reported findings in the credit accumula- 
tion domain. 

Giani et al. (2014) reported students’ persistence in college to the second year (i.e., whether students returned for 
their second year fall semester) and compared the intervention group to the propensity score matched comparison 
group sample. The authors reported, and the WWC confirmed, that there was a statistically significant difference 
between the two groups. The WWC characterizes this effect as a statistically significant positive effect. 

Struhl and Vargas (2012) reported on the number of students who had returned to college for their second year. 
These numbers were compared between intervention students and the comparison group. The authors reported, 
and the WWC confirmed, that there was a statistically significant difference between the two groups. The WWC 
characterizes this effect as a statistically significant positive effect. 

Thus, for the credit accumulation domain, two studies found statistically significant positive effects. This results in a 
rating of positive effects, with a medium to large extent of evidence. 


Summary of effectiveness for the completing high school domain 

Table 6. Rating of effectiveness and extent of evidence for the completing high school domain 


Rating of effectiveness 

Criteria met 

Positive effects 

Strong evidence of a positive effect 
with no overriding contrary evidence. 

In the two studies that reported findings, the estimated impact of the intervention on outcomes in the completing 
high school domain was positive, because three studies show statistically significant positive effects, and no 
studies show statistically significant or substantively negative effects. 

Extent of evidence 

Criteria met 

Medium to large 

Two studies that included 4,052 students in schools across the United States reported evidence of effectiveness 
in the completing high school domain. 


Two studies that meet WWC group design standards without reservations reported findings in the completing high 
school domain. 

Berger et al. (2014) reported student graduation rates for early college high school participants as compared to 
students in the comparison group. The authors reported, and the WWC confirmed, that there was a statistically sig- 
nificant difference between intervention and comparison group students on high school graduation rates. The WWC 
characterizes this finding as a statistically significant positive effect. 

Edmunds et al. (2015) reported the 5-year graduation rate for intervention students as compared to comparison 
group students. The authors reported, and the WWC confirmed, that there was a statistically significant difference 
between intervention and comparison group students on high school graduation rates. The WWC characterizes this 
finding as a statistically significant positive effect. 
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Thus, for the completing high school domain, two studies showed statistically positive effects. Dual enrollment pro- 
grams have a positive effect on completing high school with a medium to large extent of evidence. 


Summary of effectiveness for the general academic achievement (high school) domain 

Table 7. Rating of effectiveness and extent of evidence for the general academic achievement (high school) 
domain 


Rating of effectiveness 

Criteria met 

Positive effects 

Strong evidence of a positive effect 
with no overriding contrary evidence. 

In the two studies that reported findings, the estimated impact of the intervention on outcomes in the general 
academic achievement (high school) domain was positive, because two studies show statistically significant posi- 
tive effects, and no studies show statistically significant or substantively negative effects. 

Extent of evidence 

Criteria met 

Medium to large 

Two studies that included 2,817 students in schools across the United States reported evidence of effectiveness 
in the general academic achievement (high school) domain. 


Two studies that meet WWC group design standards without reservations reported findings in the general aca- 
demic achievement (high school) domain. 

Berger et al. (2014) reported on students’ standardized English language arts and math achievement scores and high 
school grade point average. The authors reported, and the WWC confirmed, that there was a statistically significant 
difference between early college high school participants and comparison participants on standardized English lan- 
guage arts assessment scores. The other two outcomes in this domain were not statistically significant. The WWC 
characterizes the mean effect across the three measures in this domain as positive and statistically significant. 

Edmunds et al. (2015) reported on the percentage of students passing the end-of-course exam in three or more 
college prep math courses, the end-of-course exam in Biology, the end-of-course exam in Civics and Economics, 
and the end-of-course exam in English I. These outcomes were obtained from transcript data collected by the North 
Carolina Department of Public Instruction (NCDPI). The authors reported, and the WWC confirmed, that there was a 
statistically significant difference between early college high school students and comparison participants. The WWC 
characterizes the mean effect across these four measures in this domain as positive and statistically significant. 

Thus, for the general academic achievement (high school) domain, two studies showed statistically significant posi- 
tive effects. Dual enrollment programs have a positive effect with a medium to large extent of evidence on general 
academic achievement (high school). 


Summary of effectiveness for the staying in high school domain 

Table 8. Rating of effectiveness and extent of evidence for the staying in high school domain 


Rating of effectiveness 

Criteria met 

Potentially positive effects 

Evidence of a positive effect with no 
overriding contrary evidence. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the staying in 
high school domain was potentially positive, because one study showed a statistically significant or substantively 
important positive effect, and no studies show a statistically significant or substantively important negative effect. 

Extent of evidence 

Criteria met 

Small 

One study that included 676 students in multiple schools in North Carolina reported evidence of effectiveness in 
the staying in high school domain. 


One study that meets WWC group design standards without reservations reported findings in the staying in high 
school domain. 
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Edmunds et al. (2015) reported the percentage of students enrolled in an early college high school who stayed in 
school versus the percentage of students enrolled in traditional high schools who stayed in school. The authors 
reported, and the WWC confirmed, that there was a statistically significant difference between the intervention stu- 
dents and the comparison students on the percentage of students who stayed in school. The WWC characterizes 
this finding as a statistically significant positive effect. 

Thus, for the staying in high school domain, one study showed a statistically positive effect. Dual enrollment pro- 
grams have a potentially positive effect on staying in high school with a small extent of evidence. 


Summary of effectiveness for the college readiness domain 

Table 9. Rating of effectiveness and extent of evidence for the college readiness domain 


Rating of effectiveness 

Criteria met 

Potentially positive effects 

Evidence of a positive effect with no 
overriding contrary evidence. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the college 
readiness domain was potentially positive, because one study showed a statistically significant or substantively 
important positive effect, and no studies show a statistically significant or substantively important negative effect. 

Extent of evidence 

Criteria met 

Small 

One study that included 1 ,355 students in multiple schools in North Carolina reported evidence of effectiveness in 
the college readiness domain. 


One study that meets WWC group design standards without reservations reported findings in the college readi- 
ness domain. 

Edmunds et al. (2015) reported the percentage of students on track to complete college preparatory coursework 
at the end of high school. The authors reported, and the WWC confirmed, that there was a statistically significant 
difference between the intervention students and the comparison students on the percentage of students who 
were ready for college at the end of high school. The WWC characterizes this finding as a statistically significant 
positive effect. 

Thus, for the college readiness domain, one study showed a statistically positive effect. Dual enrollment programs 
have a potentially positive effect on college readiness with a small extent of evidence. 


Summary of effectiveness for the attendance (high school) domain 

Table 10. Rating of effectiveness and extent of evidence for the attendance (high school) domain 


Rating of effectiveness 

Criteria met 

Potentially positive effects 

Evidence of a positive effect with no 
overriding contrary evidence. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the attendance 
(high school) domain was potentially positive, because one study showed a statistically significant or substantively 
important positive effect, and no studies show a statistically significant or substantively important negative effect. 

Extent of evidence 

Criteria met 

Small 

One study that included 1 ,554 students in 12 schools reported evidence of effectiveness in attendance (high 
school) domain. 


One study that meets WWC group design standards without reservations reported findings in the attendance (high 
school) domain. 

Edmunds et al. (2015) reported the number of days students who attended early college high schools were absent 
from school compared to students who attended traditional high schools. The authors reported, and the WWC con- 
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firmed, that there was a statistically significant difference between the intervention students and the comparison stu- 
dents on the number of days absent. The WWC characterizes this finding as a statistically significant positive effect. 

Thus, for the attendance (high school) domain, one study showed a statistically positive effect. Dual enrollment 
programs have a potentially positive effect on high school attendance with a small extent of evidence. 


Summary of effectiveness for the general academic achievement (college) domain 

Table 11. Rating of effectiveness and extent of evidence for the general academic achievement (college) 

domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

None of the studies show 
a statistically significant or 
substantively important effect, either 
positive or negative. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the general 
academic achievement (college) domain was neither statistically significant nor large enough to be substan- 
tively important. 

Extent of evidence 

Criteria met 

Small 

One study that included 455 students in 282 schools reported evidence of effectiveness in the general academic 
achievement (college) domain. 


One study that meets WWC group design standards without reservations reported findings in the general academic 
achievement (college) domain. 

Berger et al. (2014) reported students’ grade point averages (GPAs) earned in college. The authors compared col- 
lege GPAs between students in the intervention group and those in the comparison group. The authors reported, 
and the WWC confirmed, that there was not a statistically significant difference between intervention and compari- 
son group students in the general academic achievement (college) domain. The WWC characterizes this finding as 
indeterminate. The mean effect reported is neither statistically significant nor substantively important. 

Thus, for the general academic achievement (college) domain, one study showed an indeterminate effect. This 
results in a rating of no discernible effects, with a small extent of evidence. 
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Bernstein, L., Edmunds, J., & Unlu, F. (2014, April). Catching up underprepared students in Early College High 
Schools: Reducing the performance gap. Paper presented at the annual meeting of the American Educa- 
tional Research Association, Philadelphia, PA. 

Bernstein, L., Yamaguchi, R., Unlu, F., Edmunds, J., Glennie, E., Willse, J., ... Dallas, A. (2010, March). Early 
findings from the implementation and impact study of Early College High Schoo\. Paper presented at the 
Society for Research on Educational Effectiveness (SREE) conference, Washington DC. Retrieved from 
http://eric.ed. gov/?id=ED51 2692 

Edmunds, J., Unlu, F., Glennie, E., Bernstein, L., Fesler, L., Furey, J., & Arshavsky, N. (2015, November). Facili- 
tating the transition to postsecondary education: The impact of the Early College Model. Paper presented 
at the Association of Public Policy Analysis and Management conference, Miami, FL. 

Edmunds, J., Unlu, F., Glennie, E., & Fesler, L. (2015, November). Facilitating the transition to postsecondary 
education: The impact of early colleges. Paper presented at the Association for Public Policy Analysis 
and Management conference, Miami, FL. 

Edmunds, J. A. (2012). Early Colleges: A new model of schooling focusing on college readiness. New Direc- 
tions for Higher Education, 158, 81-89. 

Edmunds, J. A., Arshavsky, N., & Fesler, L. (2015, April). A mixed methods examination of college readiness 
in an innovative high school setting. Paper presented at the annual meeting of the American Educational 
Research Association, Chicago, IL. 

Edmunds, J. A., Bernstein, L., Glennie, E., Willse, J., Arshavsky, N., Unlu, F., ... Dallas, A. (2010). Preparing 
students for college: The implementation and impact of the Early College High School Model. Peabody 
Journal of Education, 85(3), 348-364. 

Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., & Arshavsky, N. (201 1 , March). The impact of the Early Col- 
lege High School Model on core 9th and 10th grade student outcomes. Paper presented at the Society 
for Research on Educational Effectiveness (SREE) conference, Washington, DC. Retrieved from http:// 
eric.ed.gov/?id=ED51 81 87 

Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., & Smith A. (2013, April). Graduating on-time: The impact of 
an innovative high school reform model on high school graduation rates. Paper presented at the annual 
meeting of the American Educational Research Association, San Francisco, CA. 
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Edmunds, J. A., Bernstein, L., Unlu, E, Glennie, E., Willse, J., Smith, A., & Arshavsky, N. (2012). Expanding the 
start of the college pipeline: Ninth-grade findings from an experimental study of the impact of the Early 
College High School Model. Journal of Research on Educational Effectiveness, 5(2), 136-159. 

Edmunds, J. A., Unlu, F., Glennie, E., Smith, A., Fesler, L, & Bernstein, L. (2013, November). The impact of 
Early College High Schools on college readiness and college enrollment. Paper presented at the annual 
meeting of the Association for Public Policy Analysis and Management, Washington, DC. 

Edmunds, J. A., Unlu, F., Smith, A., Glennie, E., & Bernstein, L. (2013, April). The impact of Early College High 
Schools on low-income students. Paper presented at the annual meeting of the American Educational 
Research Association, San Francisco, CA. 

Edmunds, J. A., Willse, J., Arshavsky, N., & Dallas, A. (2013). Mandated engagement: The impact of Early Col- 
lege High Schools. Teachers College Record, 1 15(7). 

Edmunds, J. A., Willse, J., Unlu, F., Glennie, E., & Bernstein, L. (2014, September). Increasing high school 
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Studies that meet WWC group design standards with reservations 

An, B. P. (2013). The impact of dual enrollment on college degree attainment: Do low-SES students benefit? Educa- 
tional Evaluation and Policy Analysis, 35, 57-75. doi:1 0.31 02/01 62373712461 933 
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ondary outcomes: A quasi-experimental analysis of Texas students. High School Journal, 97(4), 200-218. 
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ed.gov/?&id=ED537253 

Studies that do not meet WWC group design standards 

Allen, D., & Dadgar, M. (2012). Does dual enrollment increase students’ success in college? Evidence from a quasi- 
experimental analysis of dual enrollment in New York City. New Directions for Higher Education, 158, 11-19. 
The study does not meet WWC group design standards because equivalence of the analytic intervention and 
comparison groups is necessary and not demonstrated. 

An, B. P. (2013). The influence of dual enrollment on academic performance and college readiness: Differences by 
socioeconomic status. Research in Higher Education, 54(4), 407-432. doi:10.1007/s1 1 162-01 2-9278-z The 
study does not meet WWC group design standards because equivalence of the analytic intervention and 
comparison groups is necessary and not demonstrated. 

Chapa, M. (201 5). An evaluation of traditional and early college high school models and their effect on college readi- 
ness in English (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
3705226). The study does not meet WWC group design standards because the measures of effectiveness 
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Journal, 25, 1-5. Retrieved from http://eric.ed. gov/?&id=EJ1 055338 The study does not meet WWC group 
design standards because equivalence of the analytic intervention and comparison groups is necessary and 
not demonstrated. 

Cowan, J., & Goldhaber, D. (2014). How much of a “Running Start” do dual enrollment programs provide students? 
(CEDR Working Paper 2014-7). Seattle: University of Washington. The study does not meet WWC group 
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design standards because equivalence of the analytic intervention and comparison groups is necessary and 
not demonstrated. 

Curry, P. D. (2013). A quantitative study of the impact of early college high schools on high school dropout rates in 
Texas (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3575061) 
The study does not meet WWC group design standards because equivalence of the analytic intervention and 
comparison groups is necessary and not demonstrated. 
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the analytic intervention and comparison groups is necessary and not demonstrated. 
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persistence to the second college year (Doctoral dissertation). Available from ProQuest Dissertations and The- 
ses database. (UMI No. 3438692) The study does not meet WWC group design standards because equiva- 
lence of the analytic intervention and comparison groups is necessary and not demonstrated. 

Fowler, M. D. (2007). A program evaluation of achieving a college education plus (Doctoral dissertation). Available 
from ProQuest Dissertations and Theses database. (UMI No. 3296083) The study does not meet WWC group 
design standards because equivalence of the analytic intervention and comparison groups is necessary and 
not demonstrated. 

Additional source: 

Luna, G., & Fowler, M. (2011). Evaluation of achieving a college education plus: A credit-based 
transition program. Community College Journal of Research and Practice, 35(9), 673-688. 
doi:1 0.1 080/1 0668920903527050 

Harrington, V. S. (2005). High school/college credit programs and university success (Doctoral dissertation). Avail- 
able from ProQuest Dissertations and Theses database. (UMI No. 31661 11) The study does not meet WWC 
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and not demonstrated. 

Kanny, M. A. (2014). Forks in the pathway? Mapping the conditional effects of dual enrollment by gender, first- 
generation status, and pre-college academic achievement on first-year student engagement and grades in 
college (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
3622646) The study does not meet WWC group design standards because equivalence of the analytic 
intervention and comparison groups is necessary and not demonstrated. 

Karp, M. M., Calcagno, J. C., Hughes, K. L., Jeong, D. W., & Bailey, T. R. (2007). The postsecondary achieve- 
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analytic intervention and comparison groups is necessary and not demonstrated. 

King, S. H. (2001 ). High school/college collaborative: The impact of a co-enrollment program on student success 
(Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3008369) The 
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study does not meet WWC group design standards because the measures of effectiveness cannot be attrib- 
uted solely to the intervention. 

Langley, J. K. (2009). An evaluative study of the GateWay Early College High School (Doctoral dissertation). Avail- 
able from ProQuest Dissertations and Theses database. (UMI No. 3391935) The study does not meet WWC 
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necessary and not demonstrated. 
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ics of first year college freshmen (Doctoral dissertation). Available from ProQuest Dissertations and Theses 
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state of Texas (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
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vention and comparison groups is necessary and not demonstrated. 
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(Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3597281) The 
study does not meet WWC group design standards because equivalence of the analytic intervention and 
comparison groups is necessary and not demonstrated. 

Munoz, M. A., Fischetti, J. C., & Prather, J. R. (2014). An early college initiative in an urban, high-poverty high 
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dents Placed at Risk, 79(1), 36-52. doi:1 0.1 080/1 0824669.201 4.927746 The study does not meet WWC group 
design standards because the measures of effectiveness cannot be attributed solely to the intervention. 

Nash, C. J. (2015). Spheres of educational opportunity: A mixed methods study examining the relationship between 
concurrent enrollment participation and students’ college transition, readiness, and success (Doctoral disser- 
tation). Available from ProQuest Dissertations and Theses database. (UMI No. 3704779) The study does not 
meet WWC group design standards because equivalence of the analytic intervention and comparison groups 
is necessary and not demonstrated. 

Neely, A. L. (2013). Influence of agricultural dual credit on student college readiness self-efficacy (Doctoral disser- 
tation). Available from ProQuest Dissertations and Theses database. (UMI No. 3607800) The study does not 
meet WWC group design standards because equivalence of the analytic intervention and comparison groups 
is necessary and not demonstrated. 

Nitzke, J. E. (2002). A longitudinal study of dual credit enrollment of high school students as an accelerator for 

degree completion (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI 
No. 3061 852) The study does not meet WWC group design standards because equivalence of the analytic 
intervention and comparison groups is necessary and not demonstrated. 

Prophete, K. S. (2013). How race, gender, and Pell status affect the persistence and degree attainment rates of 
dual enrollment students (Doctoral dissertation). Available from ProQuest Dissertations Publishing. (UMI No. 
3576258) The study does not meet WWC group design standards because equivalence of the analytic inter- 
vention and comparison groups is necessary and not demonstrated. 

Rodriguez, O., Hughes, K. L., & Belfield, C. (2012). Bridging college and careers: Using dual enrollment to enhance 
career and technical education pathways (NCPR Working Paper). New York, NY: National Center for Postsec- 
ondary Research. Retrieved from http://eric.ed. gov/?&id=ED533873 The study does not meet WWC group 
design standards because equivalence of the analytic intervention and comparison groups is necessary and 
not demonstrated. 
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Saltarelli, C. A. (2008). An examination of the relationship between early college credit and higher education 

achievement, persistence, and time to graduation, of students in South Texas (Doctoral dissertation). Available 
from ProQuest Dissertations and Theses database. (UMI No. 3332682) The study does not meet WWC group 
design standards because equivalence of the analytic intervention and comparison groups is necessary and 
not demonstrated. 

Sullivan-Ham, K. (201 0). Impact of participation in a dual enrollment program on first semester college GPA (Unpub- 
lished doctoral dissertation). Walden University, Minneapolis, MN. The study does not meet WWC group 
design standards because equivalence of the analytic intervention and comparison groups is necessary and 
not demonstrated. 

Taylor, J. L. (2015). Accelerating pathways to college: The (in)equitable effects of community college dual credit. 
Community College Review, 43(4), 355-379. The study does not meet WWC group design standards because 
equivalence of the analytic intervention and comparison groups is necessary and not demonstrated. 

Young, R. D., Jr. (201 3). Dual credit enrollment and GPA by ethnicity and gender at Texas 2-year colleges (Doctoral 
dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3571399) The study does 
not meet WWC group design standards because equivalence of the analytic intervention and comparison 
groups is necessary and not demonstrated. 

Studies that are ineligible for review using the Transition to College Evidence Review Protocol 

Adelman, N., Berger, A. R., Cassidy, L., Cole, S., Duffy, H., Edwards, S Suk Yoon, K. (2009). Fifth annual Early 

College High School Initiative evaluation synthesis report. Six years and counting: The ECHSI matures. Wash- 
ington, DC & Arlington, VA: American Institutes for Research & SRI International. Retrieved from http://eric. 
ed.gov/?&id=ED514090 The study is ineligible for review because it does not have an eligible design. 

Adelman, N., Berger, A. R., Cole, S., Hall, C., Hersh, L., Knowles Keating, K Walton, L. (2005). Early College High 

School Initiative. Evaluation year end report: 2003-2004. Washington, DC & Arlington, VA: American Institutes 
for Research & SRI International. Retrieved from http://eric.ed. gov/?&id=ED494921 The study is ineligible for 
review because it does not have an eligible design. 

An, B. R, & Taylor, J. L. (2015). Are dual enrollment students college ready? Evidence from the Wabash National 
Study of Liberal Arts Education. Education Policy Analysis Archives, 23(58), 1-26. The study is ineligible for 
review because it is out of scope of the protocol. 

Anonymous. (2014). The impact of early college high schools. Community College Journal, 84(6), 36. The study is 
ineligible for review because it does not have an eligible design. 

Bruce, L. M. (2007). Perceptions, motivations, and achievement of African-American students enrolled in a middle 
college high school. (Doctoral dissertation). Available from ProQuest Dissertations. (UMI No. 3263479) The 
study is ineligible for review because it does not have an eligible design. 

Cevallos, L., & Cevallos, P. (2016). The South Los Angeles Math (SLAM) project: Year 3 report. Hacienda Heights, 
CA: College Bridge. The study is ineligible for review because it does not have an eligible design. 

Crockett-Bell, S. A. (2007). The dual credit program: Measuring the effectiveness on students’ transition from high 
school to college (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI 
No. 3277657) The study is ineligible for review because it does not have an eligible design. 

Eisenbeck Henson, M. A. (2013). Dual enrollment’s impact on college enrollment (Doctoral dissertation). Avail- 
able from ProQuest Dissertations and Theses database. (UMI No. 3567902) The study is ineligible for review 
because it does not have an eligible design. 

Farrell, T. L. (2009). The early college high school and student self-perceptions of college readiness (Doctoral disser- 
tation). Available from ProQuest Dissertations and Theses database. (UMI No. 3411426) The study is ineligible 
for review because it does not have an eligible design. 

Ganzert, B. (2014). Dual enrollment credit and college readiness. Community College Journal of Research and Prac- 
tice, 38(9), 783-793. The study is ineligible for review because it does not have an eligible design. 
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Gossman, J. (2013). Study compares admitted, non-admitted ECHS students. Education Daily, 46(125), 2. The 
study is ineligible for review because it does not have an eligible design. 

Gregory, S. W. (2009). Factors associated with Advanced Placement enrollment, Advanced Placement course 

grade, and passing of the Advanced Placement examination among Hispanic and African American students in 
Southern California (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI 
No. 3388679) The study is ineligible for review because it does not have an eligible design. 

Hall, A. L. (2008). Program implementation and student outcomes at four western North Carolina early college 
high schools (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
3319371) The study is ineligible for review because it does not have an eligible design. 

Kim, J. (2014). Relationship of Tech Prep and dual credit to college readiness and retention. College Student Jour- 
nal, 48(3), 337-346. The study is ineligible for review because it does not have an eligible design. 

Kim, J., & Bragg, D. D. (2008). The impact of dual and articulated credit on college readiness and retention in four 
community colleges. Career and Technical Education Research, 33(2), 133-158. doi:10.5328/cter33.2.133 The 
study is ineligible for review because it does not have an eligible design. 

Loftin, T. A. (2012). Concurrent and dual credit: The bridge to postsecondary education for first-generation col- 
lege students (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
3522650) The study is ineligible for review because it does not have an eligible design. 

Munoz, M. D. (2011). Early College High Schools established from 2006 through 2008 in El Paso County, Texas: 
Discovering factors contributing to Hispanic student success (Doctoral dissertation). Available from ProQuest 
Dissertations and Theses database. (UMI No. 3534116) The study is ineligible for review because it does not 
have an eligible design. 

Osumi, J. M. (2010). The influence of counselors and high school organization on the selection of participants fora 
dual credit program (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI 
No. 3434485) The study is ineligible for review because it does not have an eligible design. 

Payton-Adams, M. (2014). College success of Technical College Preparation and Dual Credit program students 

versus non-program students: A comparative analysis at a Midwestern Community College (Doctoral disserta- 
tion). Available from ProQuest Dissertations and Theses database. (UMI No. 3624025) The study is ineligible 
for review because it is out of scope of the protocol. 

Pollock, C. F. (2009). The impact of exposure to early college students on community college student academic and 
social integration (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI 
No. 3380380) The study is ineligible for review because it does not have an eligible design. 

Rochford, J. A. (2010). Ongoing proof: Results from the Canton Early College High School class of 2010. Canton, 
OH: Stark Education Partnership. Retrieved from http://eric.ed.gov/?&id=ED525147 The study is ineligible for 
review because it does not have an eligible design. 

Rowett, C. (2012). Exploring college readiness: The role of dual credit and SES on college persistence and stu- 
dent success (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 
3543604) The study is ineligible for review because it does not have an eligible design. 

Simms, R. O. (201 0). An analysis of the outcomes of dual enrollment participants in Kentucky Community & Technical 
Colleges: 2001-2002 to 2007-2008 (Doctoral dissertation). Available from ProQuest Dissertations and Theses 
database. (UMI No. 3447693) The study is ineligible for review because it does not have an eligible design. 

Speroni, C. (2011). Essays on the economics of high school-to-college transition programs and teacher effective- 
ness (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3450867) 
The study is ineligible for review because it does not have an eligible design. 

Additional sources: 

Speroni, C. (2011). Determinants of students’ success: the role of Advanced Placement and dual enrollment 
programs (NCPR Working Paper). New York, NY: National Center for Postsecondary Research. Retrieved 
from http://eric.ed. gov/?&id=ED527528 
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Speroni, C. (2012). High school dual enrollment programs: Are we fast-tracking students too fast? (NCPR 
brief). New York, NY: National Center for Postsecondary Research. Retrieved from http://eric. 
ed.gov/?&id=ED533867 

Thompson, C., & Onganga, K. (201 1). “Flying the plane while we build it”: A case study of an early college high 
school. The High School Journal, 94(2), 43-57. The study is ineligible for review because it does not have an 
eligible design. 

Welsh, J. F., Brake, N., & Choi, N. (2005). Student participation and performance in dual-credit courses in a reform 
environment. Community College Journal of Research and Practice, 29(3), 199-213. The study is ineligible for 
review because it does not have an eligible design. 

Williams, J. F. (2010). Early college academic performance: Studying the effects of earning college credits from 
advanced placement and dual enrollment (Doctoral dissertation). Available from ProQuest Dissertations and 
Theses database. (UMI No. 3390529) The study is ineligible for review because it does not use a sample 
aligned with the protocol. 
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Appendix A.1: Research details for Berger et al. (2014) 

Berger, A., Garet, M., Hoshen, G., Knudson, J., & Turk-Bicakci, L. (2014). Early college, early success: 
Early college high school initiative impact study. Washington, DC: American Institutes for Research. 

Additional source: 

Berger, A., Cassidy, L., Ford, J., Garet, M., Haxton, C., Hoshen, G Zeiser, K. (2013). Early college, 

early success: Early college high school initiative impact study. Washington, DC: American 
Institutes for Research. 


Table Al. Summary of findings Meets WWC group design standards without reservations 


Outcome domain 

Sample size 

Study findings 

Average improvement index 
(percentile points) 

Statistically significant 

Degree attainment (college) 

2,458 students 

+38 

Yes 

College access and enrollment 

2,458 students 

+12 

Yes 

Completing high school 

2,458 students 

+9 

Yes 

General academic 

2,141 students 

+3 

Yes 

achievement (high school) 




General academic 

455 students 

-1 

No 

achievement (college) 





Setting The early college high schools were located in five states throughout the country: five in urban 
areas, three in small towns, and two in mid-sized cities. Eight of the 10 early colleges were 
located on college campuses. Seven had a 2-year public college partner, two had a 4-year 
public college partner, and one had both. 

Study sample The sample consisted of general education high school students. About half (52%) of the early 

college group was female versus 55% of the comparison group. Minority students comprised 52% 
and 54% of the intervention and comparison groups, respectively. In addition, 31 % of the inter- 
vention group was first-generation college students, versus 34% of the comparison group. Low- 
income students comprised 47% of the intervention group and 42% of the comparison group. 


Intervention Six early colleges were district-run schools, and the remaining four were charter schools, 
group Most of the schools also had a subject matter focus in addition to providing opportunities 

to earn college credit: five had a STEM focus, and two had a teacher preparation focus. The 
early colleges offered a wide array of supports, with all ten early colleges providing tutoring, 
college preparatory information, and college access assistance that highlighted scholarships 
and other financial aid information. In addition, some of the early colleges offered advisories; 
summer, evening, and weekend classes; extended school days; and/or block scheduling. In 
terms of the college coursework, seven early colleges had course sequences that allowed 
students to earn at least 2 years of college credit, two early colleges allowed students to earn 
up to 1 year of college credit, and one early college allowed students to earn at least some 
college credit. 
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Comparison The comparison students in the study attended 272 different high schools. The comparison 
group schools were generally much larger than the early college high schools. At the comparison 
schools, Advanced Placement (AP) courses were more prevalent than dual enrollment as a 
strategy for students to earn college credit. The majority of the students who did not attend 
early colleges enrolled in larger high schools with larger minority and low-income student 
populations. Those schools provided fewer academic supports (e.g., tutoring) and a less direct 
focus on college readiness for all students. 


Outcomes and The study examined a number of outcomes, some measured in high school and some in col- 
measurement lege. The eligible high school outcomes were standardized English/language arts achievement, 
standardized math achievement, high school grade point average (GPA), and high school 
graduation. The eligible postsecondary outcomes were any college enrollment, any postsec- 
ondary degree, placement in developmental education in college, and college GPA. The data 
for the high school graduation outcomes were obtained from high school records maintained 
by a variety of local sources. High school standardized achievement tests in English/language 
arts and math were obtained from high school administrative records. High school GPAs were 
obtained from a student survey. The data for college enrollment and degree attainment (col- 
lege) outcomes were obtained from the National Student Clearinghouse (NSC). The data for 
college GPAs were obtained from a student survey. 

The study also reported supplemental findings for completing high school, college access and 
enrollment, and degree attainment (college) by gender, socioeconomic status, and ethnicity. 
The supplemental findings do not factor into the rating of the intervention’s effectiveness. For 
a more detailed description of these outcome measures, see Appendix B. 


Support for 
implementation 


All but one of the early colleges had college instructors, rather than qualified high school 
instructors, teaching college courses. No other support for implementation was reported. 


Appendix A.2: Research details for Edmunds et al. (2015) 

Edmunds, J., Unlu, F., Glennie, E., Bernstein, L., Fesler, L, Furey, J., & Arshavsky, N. (2015). Smoothing 
the transition to postsecondary education: The impact of the Early College Model. Retrieved from 
the SERVE website: http://www.serve.org/ 

Additional sources: 

Arshavsky, N., & Edmunds, J. A. (2014, April). The impact of Early College High Schools on math- 
ematics teaching and learning. Paper presented at the National Council of Teachers of Math- 
ematics Research Conference, New Orleans, LA. 

Bernstein, L., Edmunds, J., & Fesler, L. (2014). Closing the performance gap: The impact of the 
Early College High School Model on underprepared students. Evanston, IL: Society for 
Research on Educational Effectiveness. Retrieved from http://eric.ed.gov/?id=ED562689 
Bernstein, L., Edmunds, J., & Unlu, F. (2014, April). Catching up underprepared students in Early 
College High Schools: Reducing the performance gap. Paper presented at the annual meeting 
of the American Educational Research Association, Philadelphia, PA. 

Bernstein, L., Yamaguchi, R., Unlu, F., Edmunds, J., Glennie, E., Willse, J.,... Dallas, A. (2010, 

March). Early findings from the implementation and impact study of Early College High 
School. Paper presented at the Society for Research on Educational Effectiveness (SREE) 
conference, Washington, DC. Retrieved from http://eric.ed.gov/7idsED512692 
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Edmunds, J., Unlu, F., Glennie, E., Bernstein, L., Fesler, L., Furey, J., & Arshavsky, N. (2015, Novem- 
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Model. Paper presented at the Association of Public Policy Analysis and Management confer- 
ence, Miami, FL. 

Edmunds, J., Unlu, F., Glennie, E., & Fesler, L. (2015, November). Facilitating the transition to post- 
secondary education: The impact of early colleges. Paper presented at the Association for 
Public Policy Analysis and Management conference, Miami, FL. 

Edmunds, J. A. (2012). Early Colleges: A new model of schooling focusing on college readiness. 
New Directions for Higher Education, 158, 81-89. 

Edmunds, J. A., Arshavsky, N., & Fesler, L. (2015, April). A mixed methods examination of college 
readiness in an innovative high school setting. Paper presented at the annual meeting of the 
American Educational Research Association, Chicago, IL. 

Edmunds, J. A., Bernstein, L., Glennie, E., Willse, J., Arshavsky, N., Unlu, F.,... Dallas, A. (2010). Pre- 
paring students for college: The implementation and impact of the Early College High School 
Model. Peabody Journal of Education, 85(3), 348-364. 

Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., & Arshavsky, N. (2011, March). The impact of 
the Early College High School Model on core 9th and 10th grade student outcomes. Paper 
presented at the Society for Research on Educational Effectiveness (SREE) conference, 
Washington, DC. Retrieved from http://eric.ed.gov/7idsED518187 

Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., & Smith A. (2013, April). Graduating on-time: 

The impact of an innovative high school reform model on high school graduation rates. Paper 
presented at the annual meeting of the American Educational Research Association, San 
Francisco, CA. 

Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., Willse, J., Smith, A., & Arshavsky, N. (2012). 
Expanding the start of the college pipeline: Ninth-grade findings from an experimental study 
of the impact of the Early College High School Model. Journal of Research on Educational 
Effectiveness, 5(2), 136-159. 

Edmunds, J. A., Unlu, F., Glennie, E., Smith, A., Fesler, L., & Bernstein, L. (2013, November). The 
impact of Early College High Schools on college readiness and college enrollment. Paper pre- 
sented at the annual meeting of the Association for Public Policy Analysis and Management, 
Washington, DC. 

Edmunds, J. A., Unlu, F., Smith, A., Glennie, E., & Bernstein, L. (2013, April). The impact of Early 
College High Schools on low-income students. Paper presented at the annual meeting of the 
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Early College High Schools. Teachers College Record, 115(7). 
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tional Effectiveness, Washington, DC. 
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Table A2. Summary of findings 


Meets WWC group design standards without reservations 



Study findings 

Outcome domain 

Sample size 

Average improvement index 
(percentile points) 

Statistically significant 

Degree attainment (college) 

1,651 students 

+42 

Yes 

College access and enrollment 

676 students 

+16 

Yes 

Completing high school 

1 ,594 students 

+5 

Yes 

General academic 
achievement (high school) 

676 students 

+13 

Yes 

Staying in high school 

676 students 

+16 

Yes 

College readiness 

1,355 students 

+14 

Yes 

Attendance (high school) 

1,554 students 

+8 

Yes 


Setting This study took place in school districts throughout the state of North Carolina, including 
schools located in rural and urban settings with diverse demographics. 


Study sample All participants applied to early college high schools in the eighth grade and began their Early 
College High School programs or traditional high school in the ninth grade of high school (dur- 
ing the 2005-06, 2006-07, 2007-08, and 2008-09 school years) and were followed through 
the sixth year after starting ninth grade. Eighteen cohorts of students were represented in 
this study. The final longitudinal sample included 1,651 students (938 intervention; 713 com- 
parison). The intervention group was 59% White, 28% Black, 9% Hispanic, and 41% male, 
while the comparison group was 63% White, 25% Black, 7% Hispanic, and 41 % male. In 
both groups, 41 % of the students were first-generation college students. In addition, 51 % of 
intervention group students were free/reduced-price lunch eligible versus 50% of comparison 
group students. In the intervention group, 18% of the students were underprepared in math 
(as operationalized by passing eighth-grade standardized tests) and 21% were underprepared 
in reading. In the comparison group, 22% of the students were underprepared in math and 
20% were underprepared in reading. 


Intervention North Carolina’s early college high school model includes a program of study (grades 9-12 

group or 9-13) intended to lead to an associate degree or 2 years of college credit within 4-5 years. 

Operationally, the model includes rigorous instruction, staff collaboration and professional 
development, a focus on building positive relationships between students and staff, and stu- 
dent supports. In contrast to traditional high schools, the ECHS in North Carolina are typically 
located on college campuses, are small (fewer than 400 students), have autonomous gover- 
nance, and require students to complete 2 years of college credits while in high school. 


Comparison 

group 


The comparison students were assigned to the high school they would have attended if not 
granted admission by lottery to the early college high school. 
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Outcomes and 
measurement 


The eligible outcomes in this study were taken from administrative databases maintained by the 
North Carolina Department of Public Instruction (NCDPI), the National Student Clearinghouse, 
and the North Carolina Community College System. 

In the attendance domain (high school), the eligible primary outcome was absences (in days). 
These data were obtained from the NCDPI. 

In the college readiness domain, the primary outcome was the percentage of students on 
track to complete college preparatory coursework at the end of high school. The supplemen- 
tary outcomes in this domain included on track percentages in English, mathematics, science, 
and social studies. These data were available at multiple time points, including grades 9-12. 
The study also reported on college credits earned in high school and enrollment in college 
courses in high school; these outcomes did not meet review requirements because they are 
overaligned with the intervention. College readiness data were obtained from transcript data 
collected by NCDPI. 

For the completing high school domain, the primary outcome was the 5-year high school gradu- 
ation rate. These data were obtained from the Graduate Data Verification System. 

For the general academic achievement domain (high school), the primary outcomes included 
the percentage of students passing the end-of-course exam in three or more college prep math 
courses, the end-of-course exam in Biology, the end-of-course exam in civics and economics, 
and the end-of-course exam in English I. The study also reported the percentage of students 
passing the end-of-course exam in two or more college prep math courses, the end-of-course 
exam in one or more college prep math courses, the end-of-course exam in Algebra I, the end- 
of-course exam in Algebra II, and the end-of-course exam in Geometry. These outcomes were 
not eligible for review because they describe a subset of the information contained in the primary 
outcome of passing three or more college prep math courses. These outcomes were obtained 
from transcript data collected by NCDPI. 

For the staying in school domain, the primary outcome was the percentage of students still 
enrolled in high school. The percentage of students who dropped out was also reported and is 
included as a supplementary outcome. These outcomes were obtained from transcript data col- 
lected by NCDPI. 

In the college access and enrollment domain, self-reports of the students’ plans to attend a 
4-year college was the primary outcome. The authors reported postsecondary enrollment and 
postsecondary enrollment in a 2-year or 4-year college, but these were overaligned with the 
intervention because the figures included enrollment in any postsecondary institution during or 
after high school. The data were obtained from the National Student Clearinghouse and checked 
against the NCDPI data. 

For the degree attainment (college) domain, the primary outcome was attaining a postsecondary 
credential. Data were obtained from the National Student Clearinghouse. 

For a more detailed description of these outcome measures, see Appendix B. 
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Support for 
implementation 

The ECHS in this study collaborated with their higher education partner to develop a curricu- 
lum of high school and college courses students would take to graduate with a diploma and 2 
years of transferable college credit. Two of the design principles of North Carolina’s Learn and 
Earn ECHS model (Professionalism and Leadership) also supported implementation. As part 
of the Professionalism principle, teachers received ongoing professional development, col- 
laborated with other staff members, and had collective responsibility and decision-making. As 
part of the Leadership principle, staff worked together to create a shared mission to improve 
student outcomes. 


Appendix A.3: Research details for An (2013) 

An, B. P. (2013). The impact of dual enrollment on college degree attainment: Do low-SES students 
benefit? Educational Evaluation and Policy Analysis, 35, 57-75. doi:1 0.31 02/01 6237371 2461 933 

Table A3. Summary of findings Meets WWC group design standards with reservations 


Study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Degree attainment (college) 8,800 students +6 Yes 


Setting 

Students in the dual enrollment and comparison conditions were participants in the National 


Education Longitudinal Study (NELS), which began in 1988 with a nationally representative 
sample of eighth-grade students attending school in the United States. 

Study sample 

The intervention group contained 880 students who had participated in dual enrollment during 
high school. The comparison group contained 7,920 students who had not participated in dual 
enrollment during high school. These students came from many schools across the United 

States. No sample characteristics were provided in the study. 

Intervention 

group 

The intervention group was comprised of those individuals in the NELS sample who attended 
college and participated in dual enrollment programs in high school. No other information 
about the nature or characteristics of the dual enrollment programs was provided. 

Comparison 

group 

The comparison group was comprised of NELS participants who attended college but did 
not participate in a dual enrollment program in high school. The comparison students par- 
ticipated in other high school programs (e.g., traditional high school programs or Advanced 
Placement programs). 

Outcomes and 
measurement 

The study measured the outcome of obtaining any college degree. Additionally, supplemen- 
tal findings were reported based on student socioeconomic status (as measured by parental 
education level). All outcomes were derived from the NELS dataset. The supplemental findings 
do not factor into the intervention’s effectiveness. For a more detailed description of these 
outcome measures, see Appendix B. 

Support for 
implementation 

No information about training or implementation was provided in the study report. 
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Appendix A.4: Research details for Giani et al. (2014) 

Giani, M., Alexander, C., & Reyes, P. (2014). Exploring variation in the impact of dual-credit coursework 
on postsecondary outcomes: A quasi-experimental analysis of Texas students. High School Jour- 
nal, 97(4), 200-218. 

Table A4. Summary of findings Meets WWC group design standards with reservations 


Study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Degree attainment (college) 31,432 students +13 Yes 

College access and enrollment 31,432 students +14 Yes 

Credit accumulation 23,462 students +13 Yes 


Setting 

The study was set in Texas and included 31 ,432 ninth-grade public high school students in 


the 2000-01 school year. Data were drawn from the Texas Education Research Center P-20 
longitudinal data system. The study had 10 years of longitudinal follow-up data. 

Study sample 

The authors used the following sample characteristics in the propensity score matching pro- 
cedure: gender, race/ethnicity, socioeconomic status, English language learner, gifted, special 
education, vocational education, and prior achievement. However, the study does not report 
descriptive information for the total sample or for the two groups, so the demographic charac- 
teristics of the sample are unknown. 

Intervention 

group 

The intervention was defined as completion of at least one dual-enrollment course in the state 
of Texas during the junior or senior year. No specific information about the nature of the dual 
enrollment courses was provided. 

Comparison 

group 

The comparison condition was created from students who attended schools that did not have 
access to dual-credit courses in the state of Texas during their junior or senior year. 

Outcomes and 
measurement 

The study examined three primary outcomes and three supplementary outcomes (one of 
which met WWC standards). Study data were obtained from administrative records held at the 
Texas Education Research Center at the University of Texas at Austin. 

The three primary outcomes are: 

1) postsecondary access (access and enrollment domain) — defined as enrollment in any voca- 
tional/technical college, community college, public university, or private university in Texas 
within 1 year of high school graduation; 

2) postsecondary persistence (credit accumulation domain)— measured on the subsample 
of students who enrolled in a postsecondary institution and operationalized as enrolled in at 
least one of the traditional semesters (fall or spring) in the second year of possible postsec- 
ondary enrollment; 

3) postsecondary completion (attainment domain)— defined as earning any degree or certifi- 
cate within 6 years of completing high school. 
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One supplementary outcome met standards: 

4) postsecondary completion “first year post” (attainment domain) defined as earning any 
degree or certificate within 4 years of completing high school. It was not completely clear in 
the study, but this was likely measured at 4 years post-high school (p. 207) on the full sample 
of high school graduates. The findings for this supplementary outcome did not contribute to 
the intervention’s rating of effectiveness. 

For a more detailed description of these outcome measures, see Appendix B. 

Support for The study does not describe any specific supports for the implementation of dual enrollment 
implementation programs in the study. In Texas, dual enrollment programs are funded at least partly by the 

state. In addition, all districts in the state are required to provide dual enrollment opportunities. 

Appendix A.5: Research details for Struhl and Vargas (2012) 

Struhl, B., & Vargas, J. (2012). Taking college courses in high school: A strategy guide for college readi- 
ness: The college outcomes of dual enrollment in Texas. Washington, DC: Jobs for the Future. 


Retrieved from http://eric.ed. gov/?&id=ED537253 

Table A5. Summary of findings Meets WWC group design standards with reservations 




Study findings 

Outcome domain 

Sample size 

Average improvement index 
(percentile points) 

Statistically significant 

Degree attainment (college) 

32,908 students 

+13 

Yes 

College access and enrollment 

32,908 students 

+19 

Yes 

Credit accumulation 

32,908 students 

+16 

Yes 


Setting The study took place in the state of Texas. The study used data from the entire 2004 graduat- 
ing cohort of students from the state. Students came from multiple high schools and districts 
and were tracked for 6 years post-high school. 


Study sample The majority of students in both the intervention and comparison groups were White, with 
66% in both groups. Only 6% of intervention and 7% of comparison students were Black. 
Roughly 18% of students in each group were considered low-income. Less than 1 % in each 
group were limited English proficient students. 

Intervention Students in the intervention group participated in dual enrollment courses during their eleventh- 
group and twelfth-grade school years. These students earned college credit while still in high school. 


Comparison 

group 


Students in the comparison group did not earn college credit while in high school. They par- 
ticipated in regular high school courses and curricula. 


Outcomes and The study addresses three eligible outcomes including postsecondary enrollment (college 
measurement access and enrollment domain), returning to college a second year (credit accumulation 

domain), and earning a college degree (degree attainment [college] domain). Data for the study 
were provided by the Texas Education Research Center at the University of Texas at Austin. 

For a more detailed description of these outcome measures, see Appendix B. 
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Support for 
implementation 


The state of Texas passed a bill in 2006 that directed all districts to offer students the oppor- 
tunity to earn up to 12 college credits before graduating high school. This bill allocated $275 
per-student funding that could be used for this purpose. The bill allowed high schools and 
postsecondary institutions to form partnerships in order to meet these requirements. In 2007, 
the legislature passed another bill which provided funding for innovative high school design 
models, including early college high schools. These new bills resulted in an increase in the 
number of dual enrollment participants in the state of Texas. 
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Appendix B: Outcome measures for each domain 


Degree attainment (college) 

Any college degree 

The percent of students who had earned any college degree (as cited in An, 2013) was obtained from the 

National Educational Longitudinal Survey (NELS) of 1988 dataset. This was measured as a dichotomous 
outcome. The author also reported subgroup results by parental education. The supplemental findings do not 
factor into the intervention's rating of effectiveness. 

College completion 

College completion was assessed 6 years post-high school. Completion was based on students earning a 
degree at either a 2-year or 4-year public college (as cited in Struhl & Vargas, 2012). This was measured as a 
binary outcome. 

College degree attainment 

College degree attainment was obtained from the National Student Clearinghouse database. The percent 
of students who earned any type of postsecondary credential was reported (as cited in Berger et al., 2013). 
Subgroup results were reported by gender, race/ethnicity, income, and first-generation student status. First- 
generation outcomes did not meet standards and therefore are not reported. The other subgroup results are 
reported in Appendix D. The supplemental findings do not factor into the intervention’s rating of effectiveness. 

Postsecondary completion 

Postsecondary completion was defined as earning any type of degree within a specific period of time. The 
default was 6-years post-high school graduation (as cited in Giani et al., 2014). This was measured as a binary 
outcome. This study also measured one supplemental degree attainment (college) outcome: postsecondary 
completion within 4 years of completing high school. The supplemental findings do not factor into the interven- 
tion’s rating of effectiveness. 

Postsecondary credential 

To measure postsecondary credential, the study authors used data from the National Student Clearinghouse. 

This outcome was measured as the percent of students who earned a postsecondary credential (as cited 
in Edmunds et al., 2015). The study authors presented supplemental findings for first-generation, not first- 
generation, free/reduced-price lunch, not free/reduced-price lunch, minority, and not minority subsamples. The 
supplemental findings do not factor into the intervention’s rating of effectiveness. 

College access and enrollment 

Any college enrollment 

College access and enrollment was indexed by whether or not students enrolled in any type of college (as cited 
in Berger et al., 2013). This was reported as a percentage of students. Subgroup results were also reported by 
gender, race/ethnicity, income, and first-generation student status. The supplemental findings do not factor into 
the intervention’s rating of effectiveness. 

Enrollment in college 

College access and enrollment was measured as students enrolling in any 2- or 4-year college (as cited in Struhl 
& Vargas, 2012). This was measured as a binary outcome. 

Percent of students planning to attend a 
4-year college 

To measure the percent planning to attend a 4-year college, the study authors surveyed students in the sample 
(as cited in Edmunds et al,, 2015). The study authors presented this outcome for students in tenth grade 
(primary) and ninth grade (supplemental). The supplemental findings do not factor into the intervention's rating 
of effectiveness. 

Postsecondary access 

Postsecondary access was defined as enrollment in any vocational/technical college, community college, public 
university, or private university in the state within 1 year of high school graduation (as cited in Giani et al., 2014). 
This was measured as a binary outcome. 

Credit accumulation 

Persistence 

Persistence was operationalized as a student returning to college a second year (as cited in Struhl & Vargas, 
2012). This was measured as a binary outcome. 

Postsecondary persistence 

Credit accumulation was measured by postsecondary persistence of students who were enrolled in either the 
fall or spring semesters of the student’s second year of postsecondary education (as cited in Giani et al., 2014). 
This was measured as a binary outcome. 

Completing high school 

5-year high school graduation rate 

To measure high school graduation, study authors used the Graduate Data Verification System. The outcome 
was operationalized as the percentage of students who graduated high school within 5 years of enrolling in 
ninth grade (as cited in Edmunds et al., 2015). Results were also presented for the first-generation, not first- 
generation, free/reduced-price lunch, not free/reduced-price lunch, minority, and not minority subgroups. These 
are considered supplemental findings and do not factor into the intervention's rating of effectiveness. 
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High school graduation 

Completing high school was measured by the number of students earning their high school diploma as reported 
in high school records (as cited in Berger et al., 2013). This was reported as a percentage. Subgroup results 
were also reported by gender, race/ethnicity, income, and first-generation student status. The supplemental 
findings do not factor into the intervention's rating of effectiveness. 

General academic achievement 
(high school) 

Achievement in English language arts 
(ELA) 

Achievement in ELA was measured via standardized test scores in high school ELA tests. These scores are 
typically from tenth grade and are standardized using the state mean and standard deviation for each year and 
grade level (as cited in Berger et al., 2013). 

Achievement in mathematics 

Achievement in mathematics was measured via standardized test scores in high school English language arts 
tests. These scores are typically from tenth grade and are standardized using the state mean and standard 
deviation for each year and grade level (as cited in Berger et al., 2013). 

High school grade point average (GPA) 

To measure general academic achievement in high school, the study authors reported on students' high school 
GPA. High school GPA was measured via survey data and was reported as a variable between 0-4, which 
indicated a student's high school grade point average (as cited in Berger et al., 2013). 

Passed the end-of -course exam 

To measure passing end-of-course exams, the study authors used transcript data collected by the North 

Carolina Department of Public Instruction (NCDPI). The authors reported the percent of students who passed 
the end-of-course exam in several college preparatory courses, including Biology, Civics and Economics, English 

1, and passing three or more college preparatory mathematics courses (as cited in Edmunds et al., 2015). The 
study also reported the percentage of students passing the end-of-course exam in two or more college prep 
math courses, the end-of-course exam in one or more college prep math courses, the end-of-course exam in 
Algebra 1, the end-of-course exam in Algebra II, and the end-of-course exam in Geometry. These outcomes 
were not eligible for review because they describe a subset of the information contained in the primary outcome 
of passing three or more college prep math courses. 

Staying in high school 

Continued enrollment 

To measure continued enrollment in high school, the study authors used transcript data collected by NCDPI. This 
outcome was measured as the percent of students who continued to be enrolled at the time point measured 
(as cited in Edmunds et al., 2015). Subgroup results were presented for the free/reduced-price lunch, and not 
free/reduced-price lunch students. Dropout was also measured by the authors. To measure dropout, the study 
authors used transcript data collected by NCDPI. This outcome was measured as the percent of students who 
dropped out of school (as cited in Edmunds et al., 2015). This is a secondary outcome because the results are 
reported for the free/reduced-price lunch and not free/reduced-price lunch subsamples rather than the full study 
sample. These supplemental findings do not factor into the intervention’s rating of effectiveness. 

College readiness 

On track at the end of high school 

To measure college readiness, the study authors used data from the NCDPI. On track for college was defined 
as taking the required courses in order to enter the University of North Carolina system by the time of expected 
high school graduation (as cited in Edmunds et al., 2015). The authors reported this outcome at the end of 
grades 9-12, and the end of high school. The primary time point was at the end of high school. All other time 
points, subgroups (i.e., underrepresented minority, not underrepresented minority, first-generation student, not 
first-generation student, free/reduced-price lunch student, and not free/reduced-price lunch student), and the 
subject specific measures (i.e., on track in English, mathematics, science, and social studies) are secondary 
outcomes. The supplemental findings do not factor into the intervention’s rating of effectiveness. 

Attendance (high school) 

Absences (days) 

To measure absences, the study authors used administrative measures from the NCDPI. This outcome was 
operationalized as the number of absences per student during the school year (as cited in Edmunds et al., 

2015). Subgroup results for absences were reported by free/reduced-price lunch status. The supplemental 
findings do not factor into the intervention’s rating of effectiveness. 

General academic 
achievement (college) 

College GPA 

To measure college GPA, students who had completed at least one term of college post-high school were asked 
via a survey to report on their college grades (as cited in Berger et al., 2013). 
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Appendix C.1: Findings included in the rating for the degree attainment (college) domain 


Mean 

(standard deviation) 

WWC calculations 


Study Sample Intervention Comparison 

Outcome measure sample size group group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

An, 201 3 a 

Any college degree (%) High school 8,800 nr 

students 

nr 

nr 

0.14 

+6 

<.001 

Domain average for degree attainment (college) (An, 2013) 



0.14 

+6 

Statistically 

significant 

Berger etal., 2013 b 

College degree attainment High school 2,458 25 

(%) students (na) 

5 

(na) 

20 

1.15 

+38 

<.001 

Domain average for degree attainment (college) (Berger et al., 2013) 



1.15 

+38 

Statistically 

significant 

Edmunds etal., 201 5 C 

Postsecondary credential High school 1,651 30 

(%) students (na) 

4 

(na) 

26 

1.41 

+42 

<.001 

Domain average for degree attainment (college) (Edmunds et al., 2015) 



1.41 

+42 

Statistically 

significant 

Giani etal., 2014 d 

Postsecondary completion High school 31,432 nr 

students 

nr 

nr 

0.32 

+13 

<.001 

Domain average for degree attainment (college) (Giani et al., 2014) 



0.32 

+13 

Statistically 

significant 

Struhl & Vargas, 2012 e 

College completion High school 32,908 nr 

students 

nr 

nr 

0.34 

+13 

<.001 

Domain average for degree attainment (college) (Struhl & Vargas, 2012) 



0.34 

+13 

Statistically 

significant 

Domain average for degree attainment (college) across all studies 



0.67 

+25 

na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change 
in an average individual’s percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to 
two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of each study's domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding, na = not applicable, nr = not reported. 

a For An (2013), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was reported in 
the original study. Effect sizes were computed using the covariate-adjusted mean difference and standard errors reported in Table 1 of the study. This study is characterized as 
having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more information, please refer to the WWC Procedures and 
Standards Flandbook (version 3.0), p. 26. 

b For Berger et al. (2013), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was reported 
in the original study. The impact estimate reported in the table is derived from a model that adjusted for clustering of students within lotteries and included the following covariates: 
female, minority, first-generation, low-income, prior achievement in ELA, and prior achievement in mathematics. This study is characterized as having a statistically significant positive 
effect because the estimated effect is positive and statistically significant. For more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

c For Edmunds et al. (2015), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
calculated by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more 
information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 
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11 For Giani et al. (201 4), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was calculated 
by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more information, 
please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

e For Struhl and Vargas (201 2), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
calculated by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more 
information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 


Appendix C.2: Findings included in the rating for the college access and enrollment domain 


Mean 

(standard deviation) 

WWC calculations 


Study Sample Intervention Comparison 

Outcome measure sample size group group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Berger etal., 201 3 a 

Any college enrollment (%) High school 2,458 81 

students (na) 

72 

(na) 

9 

0.30 

+12 

< .001 

Domain average for college access and enrollment (Berger et al., 2013) 



0.30 

+12 

Statistically 

significant 

Edmunds etal., 2015 b 

Planning to attend a 4-year Tenth-grade 676 76 

college (%) students (na) 

62 

(na) 

14 

0.40 

+16 

.01 

Domain average for college access and enrollment (Edmunds et al., 2015) 



0.40 

+16 

Statistically 

significant 

Giani etal., 2014 c 

Postsecondary access High school 31,432 nr 

students 

nr 

nr 

0.35 

+14 

<.001 

Domain average for college access and enrollment (Giani et al., 2014) 



0.35 

+14 

Statistically 

significant 

Struhl & Vargas, 2012 d 

Enrollment in college High school 32,908 nr 

students 

nr 

nr 

0.49 

+19 

< .001 

Domain average for college access and enrollment (Struhl & Vargas, 2012) 



0.49 

+19 

Statistically 

significant 

Domain average for college access and enrollment across all studies 



0.39 

+15 

na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change 
in an average individual's percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to 
two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of each study's domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding, na = not applicable, nr = not reported. 

a For Berger et al. (201 3), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was reported 
in the original study. The impact estimate reported in the table is derived from a model that adjusted for clustering of students within lotteries and included the following covariates: 
female, minority, first-generation, low-income, prior achievement in ELA, and prior achievement in mathematics. This study is characterized as having a statistically significant positive 
effect because the estimated effect is positive and statistically significant. For more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

b For Edmunds et al. (2015), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
reported in the original study. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For 
more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

c For Giani et al. (201 4), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was calculated 
by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more information, 
please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

d For Struhl and Vargas (201 2), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
calculated by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more 
information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 
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Appendix C.3: Findings included in the rating for the credit accumulation domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Giani etal., 2014 a 

Postsecondary persistence 

College 

students 

23,462 

nr 

nr 

nr 

0.32 

+13 

< .001 


Domain average for credit accumulation (Giani et al., 2014) 0.32 +13 Statistically 

significant 


Struhl & Vargas, 2012 b 

Persistence College 32,908 nr nr nr 0.42 +16 < .001 

students 

Domain average for credit accumulation (Struhl & Vargas, 2012) 0.42 +16 Statistically 

significant 


Domain average for credit accumulation across all studies 0.37 +14 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change 
in an average individual’s percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to 
two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of each study’s domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding, na = not applicable, nr = not reported. 

a For Giani et al. (201 4), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was calculated 
by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more information, 
please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

b For Struhl and Vargas (201 2), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
calculated by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more 
information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

Appendix C.4: Findings included in the rating for the completing high school domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Berger etal., 201 3 a 

High school graduation (%) 

High school 

2,458 

86 

81 

5 

0.22 

+9 

.05 


students 


(na) 

(na) 






Domain average for completing high school (Berger etal., 2013) 0.22 +9 Statistically 

significant 


Edmunds etal.,2015 b 

5-year high school High school 1,594 85 82 3 0.13 +5 .009 

graduation rate (%) students (na) (na) 

Domain average for completing high school (Edmunds etal., 2015) 0.13 +5 Statistically 

significant 


Domain average for completing high school across all studies 0.18 +7 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change 
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in an average individual’s percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to 
two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of each study's domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding, na = not applicable. 

a For Berger et al. (201 3), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was reported 
in the original study. The impact estimate reported in the table is derived from a model that adjusted for clustering of students within lotteries and included the following covariates: 
female, minority, first-generation, low-income, prior achievement in ELA, and prior achievement in mathematics. This study is characterized as having a statistically significant positive 
effect because the estimated effect is positive and statistically significant. For more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

b For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
reported in the original study. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For 
more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

Appendix C.5: Findings included in the rating for the general academic achievement (high school) domain 


Mean 

(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect Improvement 


Outcome measure 

sample 

size 

group 

group 

difference 

size 

index 

p-value 

Berger etal., 201 3 a 

Achievement in English 

High school 

2,141 

0.37 

0.23 

0.14 

0.15 

+6 

< .001 

language arts 

students 


(nr) 

(nr) 





Achievement in mathematics 

High school 

1,628 

0.28 

0.23 

0.05 

0.06 

+2 

.19 


students 


(nr) 

(nr) 





High school grade point 

High school 

1,273 

2.98 

2.98 

-0.004 

-0.004 

0 

.94 

average 

students 


(nr) 

(nr) 





Domain average for general academic achievement (high school) (Berger et al., 2013) 


0.07 

+3 

Statistically 

significant 

Edmunds etal., 2015 b 

Passed the end-of-course 

Tenth-grade 

676 

68 

53 

15 

0.38 

+15 

<.001 

exam in Biology (%) 

students 


(na) 

(na) 





Passed the end-of-course 

Tenth-grade 

676 

80 

71 

9 

0.30 

+12 

<.001 

exam in Civics and 

Economics (%) 

students 


(na) 

(na) 





Passed the end-of-course 

Tenth-grade 

676 

91 

86 

5 

0.30 

+12 

<.001 

exam in English 1 (%) 

students 


(na) 

(na) 





Passed the end-of-course 

Tenth-grade 

676 

39 

28 

11 

0.30 

+12 

<.001 

exam in three or more college 
prep math courses (%) 

students 


(na) 

(na) 





Domain average for general academic achievement (high school) (Edmunds et al., 2015) 


0.32 

+13 

Statistically 

significant 

Domain average for general academic achievement (high school) across all studies 


0.20 

+7 

na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change 
in an average individual’s percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to 
two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of each study's domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding, na = not applicable, nr = not reported. 

a For Berger et al. (201 3), a correction for multiple comparisons was needed but did not affect whether any of the contrasts were found to be statistically significant. The effect sizes 
and p-values presented here were reported in the original study. This study is characterized as having a statistically significant positive effect because the effect for at least one 
measure within the domain is positive and statistically significant, and no effects are negative and statistically significant, accounting for multiple comparisons. For more information, 
please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 
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b For Edmunds et al. (201 5), a correction for multiple comparisons was needed but did not affect whether any of the contrasts were found to be statistically significant. The p-values 
presented here were calculated by the WWC. This study is characterized as having a statistically significant positive effect because the effect for at least one measure within the 
domain is positive and statistically significant, and no effects are negative and statistically significant, accounting for multiple comparisons. For more information, please refer to the 
WWC Procedures and Standards Handbook (version 3.0), p. 26. 

Appendix C.6: Findings included in the rating for the staying in high school domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Edmunds etal., 2015 a 

Continued enrollment (%) 

High school 

676 

94 

89 

5 

0.40 

+16 

< .001 


students 


(na) 

(na) 






Domain average for staying in high school (Edmunds etal., 2015) 0.40 +16 Statistically 

significant 


Domain average for staying in high school across all studies 0.40 +16 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are 
given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in 
an average individual’s percentile rank that can be expected if the individual is given the intervention. The statistical significance of the study’s domain average was determined by 
the WWC. Some statistics may not sum as expected due to rounding, na = not applicable. 

a For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
calculated by the WWC. This study is characterized as having a statistically significant positive effect because the estimated effect is positive and statistically significant. For more 
information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 

Appendix C.7: Findings included in the rating for the college readiness domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Edmunds etal., 201 5 a 

On track for college at end of 

High school 

1,355 

81 

70 

11 

0.37 

+14 

<.001 

high school (%) 

students 


(na) 

(na) 






Domain average for college readiness (Edmunds et al., 2015) 0.37 +14 Statistically 

significant 


Domain average for college readiness across all studies 0.37 +14 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are 
given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in 
an average individual's percentile rank that can be expected if the individual is given the intervention. The statistical significance of the study’s domain average was determined by 
the WWC. Some statistics may not sum as expected due to rounding, na = not applicable. 

a For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
reported in the original study. The data for this outcome were provided in Report 7 on pp. 1 1 -1 2. This study is characterized as having a statistically significant positive effect because 
the estimated effect is positive and statistically significant. For more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 
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Appendix C.8: Findings included in the rating for the attendance (high school) domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Edmunds etal., 2015 a 

Absences (days) 

High school 
students 

1,554 

4.70 

(5.56) 

6.30 

(7.89) 

1.60 

0.20 

+8 

<.001 


Domain average for attendance (high school) (Edmunds etal., 2015) 0.20 +8 Statistically 

significant 


Domain average for attendance (high school) across all studies 0.20 +8 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are 
given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in 
an average individual's percentile rank that can be expected if the individual is given the intervention. The statistical significance of the study’s domain average was determined by 
the WWC. Some statistics may not sum as expected due to rounding, na = not applicable. 

a For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was 
reported in the original study. Means for absences were provided in Report 16 on p. 16; the authors provided unadjusted standard deviations. This study is characterized as having a 
statistically significant positive effect because the estimated effect is positive and statistically significant. For more information, please refer to the WWC Procedures and Standards 
Handbook (version 3.0), p. 26. 


Appendix C.9: Findings included in the rating for the general academic achievement (college) domain 


Mean 

(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect Improvement 


Outcome measure 

sample 

size 

group 

group 

difference 

size 

index 

p-value 

Berger etal., 201 3 a 

College grade point average 

College 

455 

3.07 

3.09 

-0.02 

-0.024 

-1 

.82 


students 


(nr) 

(nr) 






Domain average for general academic achievement (college) (Berger et al., 2013) -0.024 -1 Not 

statistically 

significant 


Domain average for general academic achievement (college) across all studies -0.024 -1 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are 
given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in 
an average individual’s percentile rank that can be expected if the individual is given the intervention. The statistical significance of the study’s domain average was determined by 
the WWC. Some statistics may not sum as expected due to rounding, na = not applicable, nr = not reported. 

a For Berger et al. (201 3), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The effect size and p-value presented 
here were reported in the study. This study is characterized as having an indeterminate effect because the estimated effect was neither statistically significant nor large enough to be 
substantially important. For more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), p. 26. 
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Appendix D.1: Description of supplemental findings for the degree attainment (college) domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

An, 201 3 a 

Any college degree 

High school 
students with 
parent education = 
HS or less 

1,850 

nr 

nr 

nr 

0.16 

+6 

<.05 

Any college degree 

High school stu- 
dents with parent 
education = some 
college 

2,810 

nr 

nr 

nr 

0.17 

+7 

<.01 

Any college degree 

High school 
students with 
parent education = 
bachelor’s degree 

960 

nr 

nr 

nr 

0.06 

+3 

>.10 

Any college degree 

High school stu- 
dents with parent 
education = post 
bachelor’s degree 

670 

nr 

nr 

nr 

0.12 

+5 

> .10 

Berger etal., 2013 b 

College degree 
attainment (%) 

Female high school 
students 

1,263 

23 

(na) 

1 

(na) 

22 

2.05 

+48 

<.001 

College degree 
attainment (%) 

Male high school 
students 

1,193 

22 

(na) 

3 

(na) 

19 

1.34 

+41 

<.001 

College degree 
attainment (%) 

Minority high 
school students 

960 

65 

(na) 

1 

(na) 

64 

3.16 

+50 

<.001 

College degree 
attainment (%) 

White high school 
students 

851 

23 

(na) 

3 

(na) 

20 

1.37 

+42 

<.001 

College degree 
attainment (%) 

Low-income high 
school students 

1,187 

20 

(na) 

1 

(na) 

19 

1.94 

+47 

<.001 

College degree 
attainment (%) 

Not low-income 
high school 
students 

1,004 

25 

(na) 

3 

(na) 

22 

1.44 

+43 

<.001 

Edmunds etal., 2015 c 

Postsecondary 
credential (%) 

First-generation 

643 

23 

(na) 

3 

(na) 

20 

1.37 

+42 

<.001 

Postsecondary 
credential (%) 

Not first-generation 

950 

35 

(na) 

6 

(na) 

29 

1.29 

+40 

<.001 

Postsecondary 
credential (%) 

Free/reduced-price 

lunch 

790 

23 

(na) 

2 

(na) 

21 

1,63 

+45 

<.001 

Postsecondary 
credential (%) 

Not free/reduced- 
price lunch 

773 

37 

(na) 

7 

(na) 

30 

1.25 

+39 

<.001 
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Mean 

(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect Improvement 


Outcome measure 

sample 

size 

group 

group 

difference 

size 

index 

p-value 

Postsecondary 

Minority 

568 

20 

1 

19 

1.94 

+47 

<.001 

credential (%) 



(na) 

(na) 





Postsecondary 

Non-minority 

1,061 

36 

6 

30 

1.32 

+41 

<.001 

credential (%) 



(na) 

(na) 





Giani etal., 2014 d 

Postsecondary 
completion: 1-yr. post 

Propensity score 
matched sample 

31,432 

nr 

nr 

nr 

0.25 

+10 

<.001 


Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding, na = not applicable, nr = not reported. 

a For An (201 3), no corrections for clustering or multiple comparisons were needed. The p-values presented here were reported in the original study. 

b For Berger et al. (201 3), no corrections for clustering or multiple comparisons were needed. The p-values presented here were computed by the WWC using the information reported 
in Exhibit E.5. The impact estimates reported in the table are derived from models that adjusted for clustering of students within lotteries and covariates. The models were run using 
ten multiple imputation datasets; WWC attrition standards were met using unimputed sample sizes provided by the authors. The sample sizes reported in the table are the means 
across the ten imputed datasets, rounded to the nearest whole number, and were provided by the authors. The differences in impact estimates for degree attainment (college) were 
statistically significantly different for males and females, minorities and non-minorities, and low-income and non-low-income students. The study also reported on the impact of Early 
Colleges for first-generation college students versus non-first-generation students. This subgroup analysis did not meet WWC standards; attrition was high for the subgroups and suf- 
ficient information to establish baseline equivalence was not available. 

c For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons were needed. The p-values presented here were calculated by the WWC. 
d For Giani et al. (201 4), no corrections for clustering or multiple comparisons were needed. The p-value presented here was reported in the original study. 


Appendix D.2 Description of supplemental findings for the college access and enrollment domain 


Outcome measure 

Study 

sample 

Sample 

size 

Mean 

(standard deviation) 

WWC calculations 

p-value 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

Berger etal., 201 3 a 

Ever enrolled in college 

Female high school 

1,263 

81 

75 

6 

0.21 

+8 

<.001 

(%) 

students 


(na) 

(na) 





Ever enrolled in college 

Male high school 

1,193 

78 

66 

12 

0.37 

+14 

< .001 

(%) 

students 


(na) 

(na) 





Ever enrolled in college 

Minority high 

960 

80 

72 

8 

0.27 

+11 

<.001 

(%) 

school students 


(na) 

(na) 





Ever enrolled in college 

White high school 

851 

83 

73 

10 

0.36 

+14 

<.001 

(%) 

students 


(na) 

(na) 





Ever enrolled in college 

Low-income high 

1,187 

75 

64 

11 

0.32 

+12 

<.001 

<%) 

school students 


(na) 

(na) 





Ever enrolled in college 

Not low-income 

1,004 

85 

76 

9 

0.35 

+14 

<.001 

(%) 

high school 


(na) 

(na) 






students 
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Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Edmunds etal., 201 5 b 

Planning to attend a 

Ninth-grade 

1,604 

73 

70 

3 

0.09 

+4 

.08 

4-year college (%) 

students 


(na) 

(na) 






Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding, na = not applicable. 

a For Berger et al. (201 5), no corrections for clustering or multiple comparisons were needed. The p-values presented here were computed by the WWC using the information reported 
in Exhibit E.5. The impact estimates reported in the table are derived from models that adjusted for clustering of students within lotteries and covariates. The models were run using 
ten multiple imputation datasets; WWC attrition standards were met using unimputed sample sizes provided by the authors. The sample sizes reported in the table are the means 
across the ten imputed datasets, rounded to the nearest whole number, and were provided by the authors. For this domain, the differences in impact estimates were not significantly 
different for males and females, minorities and non-minorities, or low-income and non-low-income students. 

b For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons were needed. The p-values presented here were calculated by the WWC. 


Appendix D.3 Description of supplemental findings for the completing high school domain 


Outcome measure 

Study 

sample 

Sample 

size 

Mean 

(standard deviation) 

WWC calculations 

p-value 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

Berger etal., 201 3 a 

High school graduation 

Female high school 

1,263 

85 

83 

2 

0.09 

+4 

.11 

<%) 

students 


(na) 

(na) 





High school graduation 

Male high school 

1,193 

87 

78 

9 

0.38 

+15 

< .001 

(%) 

students 


(na) 

(na) 





High school graduation 

Minority high 

960 

87 

82 

5 

0.23 

+5 

<.001 

<%) 

school students 


(na) 

(na) 





High school graduation 

White high school 

851 

89 

83 

6 

0.31 

+12 

<.001 

(%) 

students 


(na) 

(na) 





High school graduation 

Low-income high 

1,187 

83 

74 

9 

0.33 

+13 

<.001 

(%) 

school students 


(na) 

(na) 





High school graduation 

Not low-income 

1,004 

89 

87 

2 

0.11 

+5 

.07 

(%) 

high school 


(na) 

(na) 






students 








Edmunds etal., 2015 b 

5-year HS graduation 

First-generation 

623 

82 

78 

4 

0.15 

+6 

.06 

rate (%) 



(na) 

(na) 





5-year HS graduation 

Not first-generation 

915 

90 

86 

4 

0.23 

+9 

.001 

rate (%) 



(na) 

(na) 





5-year HS graduation 

Free/reduced-price 

767 

84 

75 

9 

0.34 

+13 

< .001 

rate (%) 

lunch 


(na) 

(na) 





5-year HS graduation 

Not free/reduced- 

749 

92 

88 

4 

0.27 

+11 

<.001 

rate (%) 

price lunch 


(na) 

(na) 
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Outcome measure 

Study 

sample 

Sample 

size 

Mean 

(standard deviation) 

WWC calculations 


Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

5-year HS graduation 

Minority 

546 

88 

83 

5 

0.25 

+10 

.005 

rate (%) 



(na) 

(na) 





5-year HS graduation 

Non-minority 

1026 

86 

81 

5 

0.22 

+9 

<.001 

rate (%) 



(na) 

(na) 






Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding, na = not applicable. 

a For Berger et al. (201 3), no corrections for clustering or multiple comparisons were needed. The p-values presented here were computed by the WWC using the information reported 
in Exhibit E.5. The impact estimates reported in the table are derived from models that adjusted for clustering of students within lotteries and covariates. The models were run using 
ten multiple imputation datasets; WWC attrition standards were met using unimputed sample sizes provided by the authors. The sample sizes reported in the table are the means 
across the ten imputed datasets, rounded to the nearest whole number, and were provided by the authors. For completing high school graduation, the differences in impact estimates 
were not significantly different for males and females, minorities and non-minorities, or low-income and non-low-income students. 

b For Edmunds et al. (2015), no corrections for clustering or multiple comparisons were needed. The p-values presented here were calculated by the WWC. 

Appendix D.4 Description of supplemental findings for the staying in school domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Edmunds et al., 201 5 a 

Continued enrollment 
(%) 

Eleventh-grade 

free/reduced-price 

lunch 

710 

92 

(na) 

83 

(na) 

9 

0.52 

+20 

<.001 

Continued enrollment 
(%> 

Eleventh-grade not 
free/reduced-price 
lunch 

711 

95 

(na) 

89 

(na) 

6 

0.52 

+20 

<.001 

Dropout (%) 

Eleventh-grade 

free/reduced-price 

lunch 

710 

1.3 

(na) 

1.9 

(na) 

0.6 

0.23 

+9 

<.001 

Dropout (%) 

Eleventh-grade not 
free/reduced-price 

711 

0.3 

(na) 

0.6 

(na) 

0.3 

0.42 

+16 

<.001 


lunch 


Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding, na = not applicable. 

a For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons were needed. The p-values presented here were calculated by the WWC. 
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Appendix D.5 Description of supplemental findings for the college readiness domain 


Mean 

(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect Improvement 

Outcome measure sample size group group difference size index p-value 


Edmunds etal., 201 5 a 


On track for college at 

Ninth-grade 

1,355 

93 

85 

8 

0.52 

+20 

< .001 

end of ninth grade (%) 

students 


(na) 

(na) 





On track for college at 

Tenth-grade 

1,355 

89 

73 

16 

0.66 

+25 

< .001 

end of tenth grade (%) 

students 


(na) 

(na) 





On track for college at 

Eleventh-grade 

1,355 

84 

73 

11 

0.40 

+16 

< .001 

end of eleventh grade 
(%) 

students 


(na) 

(na) 





On track for college at 

Twelfth-grade 

1,355 

78 

68 

10 

0.31 

+12 

< .001 

end of twelfth grade 
(%) 

students 


(na) 

(na) 





On track for college by 

High school 

1,355 

97 

98 

-1 

-0.25 

-10 

< .001 

end of high school — 
English (%) 

students 


(na) 

(na) 





On track for college by 

High school 

1,355 

87 

75 

12 

0.49 

+19 

< .001 

end of high school — 
math (%) 

students 


(na) 

(na) 





On track for college by 

High school 

1,355 

98 

98 

0 

0 

0 

1.00 

end of high school — 
science (%) 

students 


(na) 

(na) 





On track for college by 

High school 

1,355 

99 

99 

0 

0 

0 

1.00 

end of high school — 
social studies (%) 

students 


(na) 

(na) 





On track for college by 

Minority 

466 

75 

67 

8 

0.24 

+9 

.013 

end of high school (%) 



(na) 

(na) 





On track for college by 

Non-minority 

861 

80 

68 

12 

0.38 

+15 

< .001 

end of high school (%) 



(na) 

(na) 





On track for college by 

First-generation 

502 

72 

57 

15 

0.40 

+16 

<.001 

end of high school (%) 



(na) 

(na) 





On track for college by 

Not first-generation 

814 

82 

75 

7 

0.25 

+10 

<.001 

end of high school (%) 



(na) 

(na) 





On track for college by 

Free/reduced-price 

621 

74 

59 

15 

0.41 

+16 

<.001 

end of high school (%) 

lunch 


(na) 

(na) 





On track for college by 

Not free/reduced- 

675 

83 

75 

8 

0.30 

+12 

<.001 

end of high school (%) 

price lunch 


(na) 

(na) 






Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding, na = not applicable. 

a For Edmunds et al. (201 5), no corrections for clustering or multiple comparisons were needed. The p-values presented here were calculated by the WWC. 
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Appendix D.6 Description of supplemental findings for the attendance (high school) domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Edmunds et al., 201 5 a 

Absences (days) 

Eleventh-grade 
students free/ 
reduced-price 
lunch 

710 

7.7 

(10.08) 

8.8 

(8.37) 

1.1 

0.04 

2 

.57 

Absences (days) 

Eleventh-grade 
students not free/ 
reduced-price 
lunch 

711 

5.6 

(6.37) 

7.1 

(7.22) 

1.5 

0.16 

6 

.03 


Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that meet WWC design standards with or without reservations, 
but do not factor into the determination of the intervention rating. For mean difference, effect size, and improvement index values reported in the table, a positive number favors 
the intervention group and a negative number favors the comparison group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing 
the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate 
presentation of the effect size, reflecting the change in an average individual’s percentile rank that can be expected if the individual is given the intervention. Some statistics may 
not sum as expected due to rounding. 

a For Edmunds et al. (2015), no corrections for clustering or multiple comparisons were needed. Unadjusted standard deviations were provided by the authors in response to an author 
query. The p-values presented here were calculated by the WWC. 
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Endnotes 

1 The descriptive information for this program was obtained from Hoffman et al. (2008), Hoffman (2005), Berger et al. (2015), Edmunds 
et al. (2015), An (2013), Giani et al. (2014), and Struhl and Vargas (2012). The WWC requests developers review the program descrip- 
tion sections for accuracy from their perspective. Further verification of the accuracy of the descriptive information for this program is 
beyond the scope of this review. 

2 Hoffman, N. (2005). Add and subtract: Dual enrollment as a state strategy to increase postsecondary success for underrepresented 
students. Boston, MA: Jobs for the Future; Hoffman, N., Vargas, J., & Santos, J. (2008). On ramp to college: A state policymaker’s 
guide to dual enrollment. Boston, MA: Jobs for the Future. 

3 The literature search reflects documents publicly available by March 2016. The studies in this report were reviewed using the 
Standards from the WWC Procedures and Standards Handbook (version 3.0), along with those described in the Transition to College 
review protocol (version 3.2). The evidence presented in this report is based on available research. Findings and conclusions may 
change as new research becomes available. 

4 For criteria used in the determination of the rating of effectiveness and extent of evidence, see the WWC Rating Criteria on p. 43. These 
improvement index numbers show the average and range of individual-level improvement indices for all findings across the studies. 

5 National Center for Education Statistics. (2013). Dual credit and exam-based courses in U.S. public high schools: 2010-11. Washing- 
ton, DC: Author. 

6 Giani, M., Alexander, C., & Reyes, P. (2014). Exploring variation in the impact of dual-credit coursework on postsecondary outcomes: 
A quasi-experimental analysis of Texas students. High School Journal, 97(4), 200-218. 

7 Groark, M. (n.d.). New investments expand and strengthen national network of early college high schools. Seattle, WA: Bill & Melinda 
Gates Foundation. Retrieved from http://www.gatesfoundation.org/ 
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WWC Rating Criteria 

Criteria used to determine the rating of a study 

Study rating 

Criteria 

Meets WWC group design 
standards without reservations 

A study that provides strong evidence for an intervention’s effectiveness, such as a well-implemented RCT. 

Meets WWC group design 

A study that provides weaker evidence for an intervention's effectiveness, such as a QED or an RCT with high 

standards with reservations 

attrition that has established equivalence of the analytic samples. 

Criteria used to determine the rating of effectiveness for an intervention 

Rating of effectiveness 

Criteria 

Positive effects 

Two or more studies show statistically significant positive effects, at least one of which met WWC group design 
standards for a strong design, AND 

No studies show statistically significant or substantively important negative effects. 

Potentially positive effects 

At least one study shows a statistically significant or substantively important positive effect, AND 

No studies show a statistically significant or substantively important negative effect AND fewer or the same number 
of studies show indeterminate effects than show statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows a statistically significant or substantively important positive effect AND at least one study 
shows a statistically significant or substantively important negative effect, but no more such studies than the number 
showing a statistically significant or substantively important positive effect, OR 

At least one study shows a statistically significant or substantively important effect AND more studies show an 
indeterminate effect than show a statistically significant or substantively important effect. 

Potentially negative effects 

One study shows a statistically significant or substantively important negative effect and no studies show a 
statistically significant or substantively important positive effect, OR 

Two or more studies show statistically significant or substantively important negative effects, at least one study 
shows a statistically significant or substantively important positive effect, and more studies show statistically 
significant or substantively important negative effects than show statistically significant or substantively important 
positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, at least one of which met WWC group design 
standards for a strong design, AND 

No studies show statistically significant or substantively important positive effects. 

No discernible effects 

None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Criteria used to determine the extent of evidence for an intervention 

Extent of evidence 

Criteria 

Medium to large 

The domain includes more than one study, AND 

The domain includes more than one school, AND 

The domain findings are based on a total sample size of at least 350 students, OR, assuming 25 students in a class, 
a total of at least 14 classrooms across studies. 

Small 

The domain includes only one study, OR 

The domain includes only one school, OR 

The domain findings are based on a total sample size of fewer than 350 students, AND, assuming 25 students in a 
class, a total of fewer than 14 classrooms across studies. 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Intervention 
Intervention report 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 


Single-case design 


Attrition occurs when an outcome variable is not available for all participants initially assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analytic sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent 
of evidence levels are given in the WWC Rating Criteria on p. 43. 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

An educational program, product, practice, or policy aimed at improving student outcomes. 

A summary of the findings of the highest-quality research on a given program, product, 
practice, or policy in education. The WWC searches for all research studies on an interven- 
tion, reviews each against design standards, and summarizes the findings of those that 
meet WWC design standards. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the 
research design and the magnitude, statistical significance, and consistency in findings. The 
criteria for the ratings of effectiveness are given in the WWC Rating Criteria on p. 43. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 
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Glossary of Terms 


Standard deviation The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significance Statistical significance is the probability that the difference between groups is a result of 

chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 


Substantively important a substantively important finding is one that has an effect size of 0.25 or greater, regardless 

of statistical significance. 

Systematic review a review of existing literature on a topic that is identified and reviewed using explicit meth- 
ods. A WWC systematic review has five steps: 1) developing a review protocol; 2) searching 
the literature; 3) reviewing studies, including screening studies for eligibility, reviewing the 
methodological quality of each study, and reporting on high quality studies and their find- 
ings; 4) combining findings within and across studies; and, 5) summarizing the review. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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Intervention 

Report 



Practice 

Guide 



Quick 

Review 


Single Study 
Review 



An intervention report summarizes the findings of high-quality research on a given program, practice, or policy in 
education. The WWC searches for all research studies on an intervention, reviews each against evidence standards, 
and summarizes the findings of those that meet standards. 


This intervention report was prepared for the WWC by Development Services Group under contract ED-IES-12-C-0084. 
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